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Final  Report 


Estimation  and  Specification  Error  in  Item  Response  Theory 
April  1980  to  August  1983 


This  final  report  summarizes  the  work  carried  out  at  Educational  Testing 
Service  over  the  40-month  period  starting  April  20,  1980  under  Contract  No. 
N00014-80-C-0402. 

RR-82-33  provides  hitherto  unavailable  formulas  for  the  sampling  error 
in  estimated  IRT  parameters  when  ability  and  item  parameters  are  estimated 
simultaneously  from  the  same  data,  as  is  usually  the  case.  RR-83-28  uses 
these  results  to  Investigate  the  effects  on  these  sampling  errors  due  to 
1)  number  of  items,  2)  number  of  examinees,  3)  distribution  of  ability, 

4)  number  of  anchor-test  items  when  two  groups  take  different  forms  of  a 
test.  It  appears  that  much  shorter  anchor  tests  can  be  used  than  was 
previously  thought  necessary.  New  and  improved  linking  methods  for  anchor- 
test  data  are  derived  and  tested  out  in  RR-82-25. 

RR-81-50  and  82-80  develop  and  apply  formulas  for  the  statistical  bias 
in  IRT  item  parameter  estimates  and  for  certain  interesting  functions  of 
these.  RR-81-20  and  82-5  develop  and  investigate  new  IRT  models  for  item 
response  function;  both  models  are  improvements  on  models  in  current  use. 

RR-81-48  and  81-49  provide  hitherto  unavailable  standard  errors  for  two 
commonly  used  equating  methods.  RR-83-26  compares  IRT  true-score  equating 
with  equipercentile  observed-score  equating;  it  is  reassuring  that  the  two 
equating  methods  yielded  almost  identical  results. 

RR-83-33  uses  decison  theory  to  develop  a  method  for  the  optimal  design 
of  conventional  tests  in  order  to  minimmize  expected  loss  due  to  errors  of 
measurement.  This  approach  should  be  widely  used  in  the  future  to  evaluate 
the  effectiveness  of  existing  tests. 

As  part  of  this  contract ,  the  IRT  program  LOGIST  was  converted  and 
Installed  to  run  on  the  UNIVAC  at  Brooks  Air  Force  Base. 

A  list  of  the  eleven  research  reports  follows,  showing  journal 
publication  where  appropriate. 
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AFHRL/MOE 

Brooks  AFB,  TX  78235 

1  Dr.  Alfred  R.  Fregly 
AFOSR/NL 

Bolling  AFB,  DC  20332 

1  Dr.  Genevieve  Haddad 
Program  Manager 
Life  Sciences  Directorate 
AFOSR 

Bolling  AFB,  DC  20332 

1  Dr.  T.  M.  Longridge 
AFHRL /OTE 

Williams  AFB,  AZ  85224 

1  Mr.  Randolph  Park 
AFHRL/MOAN 
Brooks  AFB,  TX  78235 

1  Dr.  Roger  Pennell 

Air  Force  Human  Resources  Laboratory 
Lowry  AFB,  CO  80230 

1  Dr.  Malcolm  Ree 
AFHRL/MP 

Brooks  AFB,  TX  78235 
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Air  Force 


Department  of  Defense 


1  3700  TCHTW/TTGHR 
2Lt  Tallarigo 
Sheppard  AFB,  TX  76311 

1  Lt .  Col  James  E.  Watson 
HQ  USAF/MPXOA 
The  Pentasgon 
Washington,  DC  20330 

1  Major  John  Welsh 
AFHRL/MOAN 
Brooks  AFB  ,  TX 

1  Dr.  Joseph  Yasatuke 
AFHRI/LRT 

Lowry  AFB,  CO  80230 


1  Military  Assistant  for  Training  and 
Personnel  Technology 

Office  of  the  Under  Secretary  of  Defens 
for  Research  &  Engineering 
Room  3D129,  The  Pentagon 
Washington,  DC  20301 

1  Dr.  Wayne  Sellman 

Office  of  the  Assistant  Secretary 
of  Defense  (MRA  &  L) 

2B269  The  Pentagon 
Washington,  DC  20301 

1  Major  Jack  Thorpe 
DARPA 

IhOO  Wilson  Blvd. 

Arlington,  VA  2220^- 


12  Defense  Technical  Information  Center 
Cameron  Station,  Bldg  5 
Alexandria,  VA  22314 
Attn:  TC 

1  Dr.  Craig  I.  Fields 

Advanced  Research  Projects  Agency 
1400  Wilson  Blvd. 

Arlington,  VA  22209 

1  Dr.  William  Graham 
Testing  Directorate 
MEPCOM/MEPCT-P 
Ft.  Sheridan,  IL  60037 

1  Jerry  Lehnus 
HQ  MEPCOM 
Attn:  MEPCT-P 
Fort  Sheridan,  IL  60037 
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Civilian  Agencies 

1  Dr.  Susan  Chipman 

Learning  and  Development 
National  Institute  of  Education 
1200  19th  Street  NW 
Washington,  DC  20208 

1  Dr.  Arthur  Helmed 
724  Brown 

U.  S.  Dept,  of  Education 
Washington,  DC  20208 

1  Dr.  Andrew  R.  Molnar 

Office  of  Scientific  and  Engineering 
Personnel  and  Education 
National  Science  Foundation 
Washington,  DC  20550 

1  Dr.  Vern  W.  Urry 
Personnel  R&D  Center 
Office  of  Personnel  Management 
1900  E  Street  NW 
Washington,  DC  20415 

1  Mr.  Thomas  A.  Warm 

U.  S.  Coast  Guard  Institute 
P.  0.  Substation  18 
Oklahoma  City,  OK  73169 

1  Dr.  Joseph  L.  Young,  Director 
Memory  &  Cognitive  Processes 
National  Science  Foundation 
Washington,  DC  20550 


Private  Sector 

1  Dr.  James  Alglna 
University  of  Florida 
Gainesville,  FL  326 

1  Dr.  Erling  B.  Andersen 
Department  of  Statistics 
Studiestraede  6 
1455  Copenhagen 
DENMARK 

1  Dr.  Isaac  Bejar 

Educational  Testing  Service 
Princeton,  NJ  08450 

1  Dr.  Menucha  Blrenbaum 
School  of  Education 
Tel  Aviv  University 
Tel  Aviv,  Ramat  Aviv  69978 
Israel 

1  Dr.  Werner  Birke 

Personalstammamt  der  Bundeswehr 
D-5000  Koeln  90 
WEST  GERMANY 

1  Dr.  R.  Darrell  Bock 
Department  of  Education 
University  of  Chicago 
Chicago,  IL  60637 

1  Dr.  Robert  Brennan 

American  College  Testing  Programs 

P.  0.  Box  168 

Iowa  City,  LA  52243 

1  Dr.  Glenn  Bryan 
6208  Poe  Road 
Bethesda,  MD  20817 

1  Dr.  Ernest  R.  Cadotte 
307  Stokely 

University  of  Tennessee 
Knoxville,  TN  37916 

1  Dr.  John  B.  Carroll 
409  Elliott  Rd. 

Chapel  Hill,  NC  27514 

1  Dr.  Norman  Cliff 
Dept,  of  Psychology 
Univ.  of  So.  California 
University  Park 
Los  Angeles,  CA  90007 
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Private  Sector 

1  Dr.  Allan  M.  Collins 

Bolt  Beranek  &  Nevman,  Inc. 

50  Moulton  Street 
Cambridge,  MA  02138 

1  Dr.  Hans  Crombag 

Education  Research  Center 
University  of  Leyden 
Boerhaavelaan  2 
2334  EN  Leyden 
The  NETHERLANDS 

1  CTB/McGraw-Hill  Library 
2500  Garden  Road 
Monterey,  CA  93940 

1  Dr.  Dattpradad  Divgi 
Syracuse  University 
Department  of  Psychology 
Syracuse,  NE  33210 

1  Dr.  Fritz  Drasgow 

Department  of  Psychology 
University  of  Illinois 
603  E.  Daniel  St. 

Champaign,  IL  61820 

1  Dr.  Susan  Ember tson 
PSYCHOLOGY  DEPARTMENT 
UNIVERSITY  OF  KANSAS 
Lawrence,  KS  66045 

1  ERIC  Facility-Acquisitions 
4833  Rugby  Avenue 
Bethesda,  MD  20014 

1  Dr.  Benjamin  A.  Fairbank,  Jr. 
McFann-Gray  &  Associates,  Inc. 

5825  Callaghan 
Suite  225 

San  Antonio,  TX  78228 

1  Dr.  Leonard  Feldt 

Lindquist  Center  for  Measurnent 
University  of  Iowa 
Iowa  City,  lA  52242 

1  Dr.  Richard  L.  Ferguson 

The  American  College  Testing  Program 

P.O.  Box  168 

Iowa  City,  lA  52240 


Private  Sector 

1  Univ.  Prof.  Dr.  Gerhard  Fischer 
Lleblggasse  5/3 
A  1010  Vienna 
AUSTRIA 

1  Professor  Donald  Fitzgerald 
University  of  New  England 
Armidale,  New  South  Wales  2351 
AUSTRALIA 

1  Dr.  Dexter  Fletcher 
University  of  Oregon 
Department  of  Computer  Science 
Eugene,  OR  97403 

1  Dr.  John  R.  Frederiksen 
Bolt  Beranek  &  Newman 
50  Moulton  Street 
Cambridge,  MA  02138 

1  Dr.  Janice  Gifford 

University  of  Massachusetts 
School  of  Education 
Amherst,  MA  01002 

1  Dr.  Robert  Glaser 

Learning  Research  &  Development  Center 
University  of  Pittsburgh 
3939  O'Hara  Street 
PITTSBURGH,  PA  15260 

1  Dr.  Bert  Green 

Johns  Hopkins  University 
Department  of  Psychology 
Charles  &  34th  Street 
Baltimore,  MD  21218 

1  Dr.  Ron  Hambleton 
School  of  Education 
University  of  Massachusetts 
Amherst,  MA  01002 

1  Dr.  Paul  Horst 
677  G  Street,  #184 
Chula  Vista,  CA  90010 

1  Dr.  Lloyd  Humphreys 
Department  of  Psychology 
University  of  Illinois 
603  East  Daniel  Street 
Champaign,  IL  61820 


Educational  Testing  Servlce/Lord  (NR  150-453)  20-0ct-83 


Page 


8 


Private  Sector 

1  Dr.  Earl  Hunt 

Dept,  of  Psychology 
University  of  Washington 
Seattle,  WA  98105 

1  Dr.  Jack  Hunter 
2122  Coolldge  St. 

Lansing,  MI  48906 

1  Dr.  Huynh  Huynh 
College  of  Education 
University  of  South  Carolina 
Columbia,  SC  29208 

1  Dr.  Douglas  H.  Jones 

Advanced  Statistical  Technologies 
Corporation 
10  Trafalgar  Court 
Lawrencevllle ,  NJ  08148 

1  Professor  John  A.  Keats 
Department  of  Psychology 
The  University  of  Newcastle 
N.S.W.  2308 
AUSTRALIA 

1  Dr.  William  Koch 

University  of  Texas-Austln 
Measurement  and  Evaluation  Center 
Austin,  TX  78703 

1  Dr.  Marcy  Lansman 

The  L.  L.  Thurstone  Psychometric 
Laboratory 

University  of  North  Carolina 
Davie  Hall  013A 
Chapel  Hill,  NC  27514 

1  Dr.  Alan  Lesgold 
Learning  R&D  Center 
University  of  Pittsburgh 
3939  O'Hara  Street 
Pittsburgh,  PA  15260 

1  Dr.  Michael  Levine 

Department  of  Educational  Psychology 
210  Education  Bldg. 

University  of  Illinois 
Champaign,  IL  61801 


Private  Sector 

1  Dr.  Charles  Lewis 

Facultelt  Soclale  Wetenschappen 
Rljksunlversltelt  Groningen 
Oude  Boterlngestraat  23 
9712GC  Groningen 
Netherlands 

1  Dr.  Robert  Linn 

College  of  Education 
University  of  Illinois 
Urbana,  IL  61801 

1  Mr.  Phillip  Livingston 

Systems  and  Applied  Sciences  Corporatlo 
6811  Kenilworth  Avenue 
Rlverdale,  MD  20840 

1  Dr.  Robert  Lockman 

Center  for  Naval  Analysis 
200  North  Beauregard  St. 

Alexandria,  VA  22311 

1  Dr.  Frederic  M.  Lord 

Educational  Testing  Service 
Princeton,  NJ  08541 

1  Dr.  James  Lumsden 
Department  of  Psychology 
University  of  Western  Australia 
Nedlands  W.A.  6009 
AUSTRALIA 

1  Dr.  Gary  Marco 
Stop  31-E 

Educational  Testing  Service 
Princeton,  NJ  08451 

1  Dr.  Scott  Maxwell 

Department  of  Psychology 
University  of  Notre  Dame 
Notre  Dame,  IN  46556 

1  Dr.  Samuel  T.  Mayo 

Loyola  University  of  Chicago 
820  North  Michigan  Avenue 
Chicago,  IL  60611 

1  Mr.  Robert  McKinley 

American  College  Testing  Programs 

P.O.  Box  168 

Iowa  City,  lA  52243 
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Private  Sector 

1  Dr.  Barbara  Means 

Human  Resources  Research  Organization 
300  North  Washington 
Alexandria,  VA  22314 

1  Dr.  Robert  Mislevy 
711  Illinois  Street 
Geneva,  IL  60134 

1  Dr.  Allen  Munro 

Behavioral  Technology  Laboratories 
1845  Elena  Ave. ,  Fourth  Floor 
Redondo  Beach,  CA  90277 

1  Dr.  W.  Alan  Nlcewander 
University  of  Oklahoma 
Department  of  Psychology 
Oklahoma  City,  OK  73069 

1  Dr.  Donald  A  Norman 
Cognitive  Science,  C-015 
Univ.  of  California,  San  Diego 
La  Jolla,  CA  92093 

1  Dr,  Melvin  R.  Novick 

356  Lindquist  Center  for  Measurment 
University  of  Iowa 
Iowa  City,  lA  52242 

1  Dr.  James  Olson 
WICAT,  Inc. 

1875  South  State  Street 
Orem,  UT  84057 

1  Wayne  M.  Patience 

American  Council  on  Education 
GED  Testing  Service,  Suite  20 
One  Dupont  Cirle,  NW 
Washington,  DC  20036 

1  Dr.  James  A.  Paulson 
Portland  State  University 
P.O.  Box  751 
Portland,  OR  97207 

1  Dr.  James  W.  Pellegrino 
University  of  California, 

Santa  Barbara 
Dept,  of  Psychology 
Santa  Barabara,  CA  93106 


Private  Sector 

1  Dr.  Mark  D.  Reckase 
ACT 

P.  0.  Box  168 
Iowa  City,  lA  52243 

1  Dr.  Lauren  Resnick 
LRDC 

University  of  Pittsburgh 
3939  O'Hara  Street 
Pittsburgh,  PA  1521 

1  Dr.  Thomas  Reynolds 

University  of  Texas-Dallas 
Marketing  Department 
P.  0.  Box  688 
Richardson,  TX  75080 

1  Dr.  Andrew  M.  Rose 

American  Institutes  for  Research 
1055  Thomas  Jefferson  St.  NW 
Washington,  DC  20007 

1  Dr.  Lawrence  Rudner 
403  Elm  Avenue 
Takoma  Park,  MD  20012 

1  Dr.  J.  Ryan 

Department  of  Education 
University  of  South  Carolina 
Columbia,  SC  29208 

1  PROF.  FUMIKO  SAMEJIMA 
DEPT.  OF  PSYCHOLOGY 
UNIVERSITY  OF  TENNESSEE 
KNOXVILLE,  TN  37916 

1  Dr.  Walter  Schneider 
Psychology  Department 
603  E.  Daniel 
Champaign,  IL  61820 

1  Lowell  Schoer 

Psychological  &  Quantitative 
Foundations 
College  of  Education 
University  of  Iowa 
Iowa  City,  lA  52242 

1  Dr.  Kazuo  Shigemasu 
7-9-24  Kugenuma-Kaigan 
Fujusawa  251 
JAPAN 
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1  Dr.  Edwin  Shirkey 

Department  of  Psychology 
University  of  Central  Florida 
Orlando,  FL  32816 

1  Dr.  William  Sims 

Center  for  Naval  Analysis 
200  North  Beauregard  Street 
Alexandria,  VA  22311 

1  Dr.  H.  Wallace  Slnalko 
Program  Director 

Manpower  Research  and  Advisory  Services 
Smithsonian  Institution 
801  North  Pitt  Street 
Alexandria,  VA  22314 

1  Dr.  Kathryn  T.  Spoehr 
Psychology  Department 
Brown  University 
Providence,  RI  02912 

1  Dr.  Robert  Sternberg 
Dept,  of  Psychology 
Yale  University 
Box  llA,  Yale  Station 
New  Haven,  CT  06520 

1  Dr.  Peter  Stoloff 

Center  for  Naval  Analysis 
200  North  Beauregard  Street 
Alexandria,  VA  22311 

1  Dr.  William  Stout 
University  of  Illinois 
Department  of  Mathematics 
Urbana,  IL  61801 

1  DR.  PATRICK  SUPPES 

INSTITUTE  FOR  MATHEMATICAL  STUDIES  IN 
THE  SOCIAL  SCIENCES 
STANFORD  UNIVERSITY 
STANFORD,  CA  94305 

1  Dr.  Harlharan  Swamlnathan 
Laboratory  of  Psychometric  and 
Evaluation  Research 
School  of  Education 
University  of  Massachusetts 
Amherst,  MA  01003 


1  Dr.  Kikuml  Tatsuoka 

Computer  Based  Education  Research  Lab 
252  Engineering  Research  Laboratory 
Urbana,  IL  61801 

1  Dr.  Maurice  Tatsuoka 
220  Education  Bldg 
1310  S.  Sixth  St. 

Champaign,  IL  61820 

1  Dr.  David  Thissen 

Department  of  Psychology 
University  of  Kansas 
Lawrence ,  KS  66044 

1  Dr.  Douglas  Towne 

Univ.  of  So.  California 
Behavioral  Technology  Labs 
1845  S.  Elena  Ave. 

Redondo  Beach,  CA  90277 

1  Dr.  Robert  Tsutakawa 
Department  of  Statistics 
University  of  Missouri 
Columbia,  MO  65201 

1  Dr.  J.  Uhlaner 
Uhlaner  Consultants 
4258  Bonavita  Drive 
Encino,  CA  91436 

1  Dr.  V.  R.  R.  Uppuluri 
Union  Carbide  Corporation 
Nuclear  Division 
P.  0.  Box  Y 
Oak  Ridge,  TN  37830 

1  Dr.  David  Vale 

Assessment  Systems  Corporation 
2233  University  Avenue 
Suite  310 

St.  Paul,  MN  55114 

1  Dr.  Kurt  Van  Lehn 
Xerox  PARC 

3333  Coyote  Hill  Road 
Palo  Alto,  CA  94304 


1  Dr.  Howard  Wainer 

Division  of  Psychological  Studies 
Educational  Testing  Service 
Princeton,  NJ  08540 
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Private  Sector 

1  Dr.  Michael  T.  Waller 

Department  of  Educational  Psychology 
University  of  Wisconsin — Milwaukee 
Milwaukee,  WI  53201 

1  Dr.  Brian  Waters 
HumRRO 

300  North  Washington 
Alexandria,  VA  22314 

1  Dr.  Phyllis  Weaver 
2979  Alexis  Drive 
Palo  Alto,  CA  94304 

1  Dr.  David  J.  Weiss 
N660  Elliott  Hall 
University  of  Minnesota 
75  E.  River  Road 
Minneapolis,  MN  55455 

1  Dr.  Donald  0.  Weltzman 
Mitre  Corporation 
1820  Dolley  Madison  Blvd 
McLean,  VA  22102 

1  Dr.  Rand  R.  Wilcox 

University  of  Southern  California 
Department  of  Psychology 
Los  Angeles,  CA  90007 

1  Wolfgang  Wildgrube 
Streitkraef teamt 
Box  20  50  03 
D-5300  Bonn  2 
WEST  GERMANY 

1  Dr.  Bruce  Williams 

Department  of  Educational  Psychology 
University  of  Illinois 
Urbana,  XL  61801 

1  Dr.  Wendy  Yen 
CTB/McGraw  Hill 
Del  Monte  Research  Park 
Monterey,  CA  93940 
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